Papers with entailment models
Label Verbalization and Entailment for Effective Zero and Few-Shot Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Relation extraction systems require large amounts of labeled examples which are costly to annotate. |
| Approach: | They propose to use hand-made relation extraction tasks to refine a pretrained textual entailment engine which is run as-is or further fine-tuned on labeled examples. |
| Outcome: | The proposed system achieves 63% F1 zero-shot, 69% with 16 examples per relation and 4 points short of the state-of-the-art system on the same conditions. |
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | Recent advances on abstractive summarization have led to fluent summaries, but factual errors in generated summary still severely limit their use in practice. |
| Approach: | They evaluate summaries produced by state-of-the-art models via crowdsourcing and show that factual errors occur frequently. |
| Outcome: | The proposed models can detect errors and reduce them by reranking alternative summaries. |
Repurposing Entailment for Multi-Hop Question Answering Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to use entailment models for question answering are limited . large scale datasets are typically framed at a sentence level, whereas question answering requires verifying whether multiple sentences, taken together as a premise, entitle a hypothesis. |
| Approach: | They propose a general architecture that can use entailment models for multi-hop QA tasks. |
| Outcome: | The proposed model outperforms QA models trained on target datasets and the OpenAI transformer models. |
Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods of offensive text detection perform poorly when asked to detect implicitly offensive statements . a dataset based on SLIGHT provides a framework for implicit offensive text identification . |
| Approach: | They propose a dataset to support the task of implicit offensive text detection in dialogues . they show that reasoning is crucial for understanding this broader class of offensive utterances - SLIGHT . |
| Outcome: | The proposed model achieves 11% accuracy in implicit offensive text detection tasks . the proposed model can be used to identify toxic speech in specific domains . |
WiCE: Real-World Entailment for Claims in Wikipedia (2023.emnlp-main)
Copied to clipboard
| Challenge: | Textual entailment models are increasingly used in fact-checking, presupposition verification in question answering, or summary evaluation. |
| Approach: | They propose a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia that provides en-tailment judgments over sub-sentence units of the claim and a minimal subset of evidence sentences that support each subclaim. |
| Outcome: | The proposed dataset improves on multiple datasets at test time and shows that real claims involve verification and retrieval problems that existing models fail to address. |